Differential delay compensation

ABSTRACT

In one embodiment, a method includes recording, for each of a plurality of data frames, a virtual write address including a multiframe indicator and a byte number indicator; and reading a group of associated data frames identified by corresponding multiframe indicators and byte number indicators. The reading is based on determining a minimum write address from a plurality of physical write addresses in the group of associated data frames by comparing virtual write addresses of all members in the group.

PRIORITY APPLICATION

This application is a continuation of U.S. application Ser. No.13/229,455, filed Sep. 9, 2011, now issued as U.S. Pat. No. 8,488,631,which is a continuation of U.S. application Ser. No. 11/093,907, filedMar. 30, 2005, now issued as U.S. Pat. No. 8,018,926, all of which areincorporated herein by reference in their entireties.

BACKGROUND

The subject matter described herein relates generally to the field ofelectronic communication and more particularly to differential delaycompensation.

Communication networks transmit data from an originator to a destinationvia a communication network that may include multiple transfer points,such as hardware routers, that receive data typically in the form ofpackets or data frames. Data transmission over fiber optics networks mayconform to SONET and/or SDH standards. SONET and SDH are a set ofrelated standards for synchronous data transmission over fiber opticnetworks. SONET is short for Synchronous Optical NETwork and SDH is anacronym for Synchronous Digital Hierarchy.

SONET/SDH networks may employ virtually concatenated payloads. Virtualconcatenation partitions payload data into multiple virtual containersthat may be assigned a single index, referred to as a MultiframeIndicator (MFI), and transmitted contemporaneously across differenttransmission media and/or different network paths. Because the payloaddata traverses different network paths, payload data transmittedcontemporaneously can be received at different times, an effect referredto as differential delay. Differential delay can also result frompointer processing, or from other considerations.

Virtual concatenation compensates for differential delay at thereceiving entity by reassembling the payload in an appropriatetime-ordered sequence. Data from different members of a virtualconcatenation group are stored in a memory at the receiver. Processinglogic in the destination node reads payload data from members having thesame MFI contemporaneously. To do this, the destination node maydetermine memory locations of members in the same group having the sameMFI value at the same byte within the frame so that the data can beassembled correctly at the output of the data memory. Addressing data inthis arrangement can be complex, as data is being received, assembled,and processed at varying points and may require data reading from orwriting to the memory based on a variety of circumstances.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures.

FIG. 1A is a schematic illustration of a SONET/SDH communication systemin accordance with one embodiment.

FIG. 1B is a schematic illustration of a suitable system in accordancewith one embodiment.

FIG. 2A is a schematic illustration of write operations into a memory ata receiver.

FIG. 2B is a schematic illustration of a memory at a receiver inaccordance with one embodiment.

FIG. 3 is a flowchart illustrating operations in one embodiment of amethod for writing data frames into a memory.

FIG. 4 is a flowchart illustrating operations in one embodiment of amethod for reading data frames from a memory.

FIG. 5 is a schematic illustration of an eight-byte wide memory with Nwords in accordance with one embodiment.

FIG. 6 is a schematic illustration of one embodiment of a memoryarchitecture in which M bytes of SDRAM store 64 STS-3c members.

DETAILED DESCRIPTION

Described herein are exemplary systems and methods for differentialdelay compensation in a communication system. In the followingdescription, numerous specific details are set forth in order to providea thorough understanding of various embodiments. However, it will beunderstood by those skilled in the art that the various embodiments maybe practiced without the specific details. In other instances,well-known methods, procedures, components, and circuits have not beendescribed in detail so as not to obscure the particular embodiments.

The methods described herein may be embodied as logic instructions on acomputer-readable medium. When executed on a processor, the logicinstructions cause a processor to be programmed as a special-purposemachine that implements the described methods. The processor, whenconfigured by the logic instructions to execute the methods describedherein, constitutes structure for performing the described methods.

FIG. 1A is a schematic illustration of a SONET/SDH communicationswitching system in accordance with one embodiment. Referring to FIG.1A, SONET/SDH switching system 100 includes a transmitter 110 connectedthrough a communication pathway 115 to a switching network 120.Switching network 120 is connected through a communication pathway 125to a destination 130.

Transmitter 110 transmits data as a series of payloads/frames to thedestination 130 through the switching network 120. Packets may passthrough a variety of hardware and/or software components, such asservers, routers, switches, etc. in transmission across switchingnetwork 120. As each frame arrives at a hardware and/or softwarecomponent, the component may store the frame briefly before transmittingthe frame to the next component. The frames proceed through the networkuntil they arrive at the destination 130. The destination 130 maycontain one or more processors 135 and/or one or more memory modules140.

FIG. 1B is a schematic illustration of a suitable system in accordancewith one embodiment. The system 101 may include a line card 111, a linecard 121, a switch fabric 141, and a backplane interface 131. Line card111 may be implemented as a SONET/SDH add-drop multiplexer, a FibreChannel compatible line input, an Ethernet line input, a SONET/SDH lineinput, or the like.

Line card 121 may be implemented as a transceiver capable oftransmitting and receiving frames and/or packets to and from a networkthat is compatible with SONET/SDH as well as other protocols such asOTN, TFI-5, and Ethernet, although other standards may be used. Forexample, SONET/SDH and OTN are described for example in: ITU-TRecommendation G.709 Interfaces for the optical transport network (OTN)(2001); ANSI T1.105, Synchronous Optical Network (SONET) BasicDescription Including Multiplex Structures, Rates, and Formats; BellcoreGeneric Requirements, GR-253-CORE, Synchronous Optical Network (SONET)Transport Systems: Common Generic Criteria (A Module of TSGR, FR-440),Issue 1, December 1994; ITU Recommendation G.872, Architecture ofOptical Transport Networks, 1999; ITU Recommendation G.825, “Control ofJitter and Wander within Digital Networks Based on SDH” March, 1993; ITURecommendation G.957, “Optical Interfaces for Equipment and SystemsRelating to SDH”, July, 1995; ITU Recommendation G.958, Digital LineSystems based on SDH for use on Optical Fibre Cables, November, 1994;and/or ITU-T Recommendation G.707, Network Node Interface for theSynchronous Digital Hierarchy (SDH) (1996). For example, animplementation of TFI-5 is described in TFI-5: TDM Fabric to FramerInterface Implementation Agreement (2003) available from the OpticalInternetworking Forum (OIF). For example, IEEE 802.3 describes Ethernetstandards.

Switching network 120 may be any network such as the Internet, anintranet, a local area network (LAN), storage area network (SAN), a widearea network (WAN). One embodiment of line card 121 may include physicallayer processor 122, framer 124, network processor 126, and host-controlplane controller 128.

Physical layer processor 122 may receive optical or electrical signalsfrom the network and prepare the signals for processing by downstreamelements such as framer 124. For example, for frames and/or packetsreceived from the network, physical layer processor 122 may convert anoptical signal to electrical format and/or remove jitter from signalsfrom the network. For frames and/or packets to be transmitted to thenetwork, physical layer processor 122 may remove jitter from signalsprovided by upstream devices such as framer 124 and prepare signals fortransmission to the network, which may be optical or electrical format.

Framer 124 may utilize techniques described herein to process framesand/or packets received from a network. Framer 124 may transfer overheadfrom frames and/or packets to a higher layer level processor such as anetwork processor 126. For example, framer 124 and network processor 126may communicate using an interface compatible for example with SPI-4(described for example in the Optical Internetworking Forum (OIFDocument) OIF-SPI4-02.1 and ITU-T G.707 2000, T1.105-2001 (draft),T1.105.02-1995, and ITU-T recommendations G.7042 and G.707), althoughinterfaces compatible with other standards may be used.

Network processor 126 may perform layer 2 or layer 3 (as well as otherhigher layer level) processing on frames and/or packets provided by andto framer 124 in conformance with applicable link, network, transportand application protocols. Network processor 126 also may performtraffic management at the IP layer.

Host-control plane controller 128 may configure operation of framer 124and network processor 126. For example, host-control plane controller128 may program/provision framer 124 to control the content of frames.Host-control plane controller 128 may be implemented as separate fromnetwork processor 126 and communicate with the framer 124 and networkprocessor 126 using an interface that complies with Peripheral ComponentInterconnect (PCI) Local Bus Specification, Revision 2.2, Dec. 18, 1998available from the PCI Special Interest Group, Portland, Oreg., U.S.A.(as well as revisions thereof) or PCI-X Specification Rev. 1.0a, Jul.24, 2000, available from the aforesaid PCI Special Interest Group,Portland, Oreg., U.S.A., although other standards may be used.Host-control plane controller 128 could be implemented as part ofnetwork processor 126, although other implementations may be used.

In one embodiment, one or more of physical layer processor 122, framer124, or network processor 126 may be coupled to volatile and/ornonvolatile memory module 127. For example, memory module 127 mayinclude one or more of the following: read-only memory (ROM),programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM(EEPROM), a disk drive, a floppy disk, a compact disk ROM (CD-ROM), adigital video disk (DVD), flash memory, a magneto-optical disk, or othertypes of nonvolatile machine-readable media suitable for storingelectronic instructions and/or data.

In one embodiment, components of line card 121 may be implemented amongthe same integrated circuit. In another embodiment, components of linecard 121 may be implemented among several integrated circuits thatcommunicate using, for example, a bus or conductive leads of a printedcircuit board.

Backplane interfaces 131 may be implemented as a single or multi-pininterface and may be used by line cards to interface with system fabric141. For example, backplane interfaces 131 may be compatible with TFI-5or CSIX (described in CSIX-L1: Common Switch Interface Specification-L1(2000)), although other standards may be used. Switch fabric 141 maytransfer IP packets or Ethernet packets (as well as other information)between line cards based on relevant address and header information.Switch fabric 141 can be implemented as a packet switch fabric or a TDMcross connect. Switch fabric 141 can be any device (or devices) thatinterconnect numerous dataplanes of subsystems (i.e., linecards)together.

SONET/SDH defines optical carrier levels and electrically equivalentsynchronous transport signals (STSs) for the fiber-optic basedhierarchy. In SONET, any type of service, ranging from voice to highspeed data and video, can be accepted by various types of serviceadapters. A service adapter maps the signal into the payload envelope ofthe STS-1 or virtual tributary. All inputs received are converted to abase format of a synchronous signal, referred to as STS-1, whichtransmits at 51.84 Mbps (or higher). Several synchronous STS-1s may bemultiplexed together to form a higher-level STS-n signal, which areinteger multiples of an STS-1 signal.

SONET networks transmit data in frames, which include a transportoverhead and a synchronous payload envelope (SPE). An SPE includes anSTS path overhead section and a payload section, which holds the databeing transported over the SONET network. Once the payload ismultiplexed into the SPE and transmitted, the payload is not examined atintermediate nodes.

SONET/SDH architecture supports virtual concatenation. In virtualconcatenation a large payload may be divided into a group of smallerpayloads, which may be transmitted contemporaneously across differentcommunication channels. Each SPE within a concatenated group contains anidentifier, called a Multi-Frame Identifier, or MFI. The MFI forms partof the SONET/SDH path overhead information in the SPE and indicates theSPE's sequence and position within the group.

Virtual concatenation does not require intermediate node support. Tocompensate for differential delay, a receiver at the destinationtemporarily stores frames in a memory so that the payload data can beproperly realigned. Applying an address calculation method that allowscontinuous data storage from members (i. e., time slots) in virtuallyconcatenated groups permits the efficient use of the memory, therebyincreasing the differential delay range which may be compensated for agiven memory size. In one embodiment, a 1-byte wide on-chip memory isused as an example. The scheme can be extended for wider memory or forexternal memory modules. On-chip memory or external RAM modules can beused as a memory.

Exemplary operations for writing received data frames into a memory areexplained with reference to FIG. 2A and FIG. 3. FIG. 2A is a schematicillustration of a memory at a receiver, and FIG. 3 is a flowchartillustrating operations in one embodiment of a method for writing dataframes into a memory such as the buffer illustrated in FIG. 2A.Referring to FIG. 3, at operation 310 data frames are received at adestination node in a communication network. At operation 315 thepayload data from the received data frames are stored in a memory. Atoperation 320 the physical write address at which a received frame iswritten in memory is recorded in a suitable memory, for example aflip-flop or other memory device. In one embodiment the physical writeaddress corresponds to a physical location in the memory. At operation325 a virtual write address is recorded in a suitable memory, forexample a flip-flop or other memory device. In one embodiment thevirtual write address includes the MFI value associated with thereceived data frame and the byte number for the last received byte.

As illustrated in FIG. 2A, the payload data frames received at thereceiver may be written into the memory continuously. The physical writeaddress is incremented continuously as frames are received. FIG. 2Aillustrates an embodiment in which a 2.5 KB (2560 bytes) memory is usedto store STS-3c payloads, which include 2349 bytes per frame. In thisexample the J1 byte for a given frame i is stored at address 130, andthe J1 byte for frame i+1 and frame i+2 are stored at address 2479 and2268, respectively. Writing data to the memory in a continuous fashionmakes efficient use of the memory.

To compensate for differential delay, data frames from different membershaving the same MFI and same byte number may be read at the same time.Exemplary operations for reading received data frames into a memory areexplained with reference to FIG. 2B and FIG. 4. FIG. 2B is a schematicillustration of a memory at a receiver, and FIG. 4 is a flowchartillustrating operations in one embodiment of a method for reading dataframes from a memory such as the buffer illustrated in FIG. 2B.

Referring to FIG. 4, at operation 410 a minimum write address for agroup is determined from the virtual write addresses of all membersbelong to this group. In one embodiment the minimum write address may bedetermined by comparing the virtual write addresses of all members inthe same group, and selecting the minimum write address. The minimumwrite address may be expressed using a multiframe indicator and a bytenumber.

At operation 415 a virtual read address is determined using the minimumwrite address. In one embodiment the virtual read address is determinedby subtracting a threshold value equal to the write to read delay inmemory from the minimum write address.

At operation 420 a physical read address is determined using the virtualread address. In one embodiment the physical read address for eachmember in a group may be determined using the relationship:RAPhy(i)=WAPhy(i)−(WAVir(i)−RAVir)where RAPhy (i) is the physical read address for member i, WAPhy (i) isthe physical write address for member i, WAVir (i) is the virtual writeaddress for member i, and RAVir is the virtual read address for thewhole group.

Although the virtual read address is the same for the whole group, dueto the address conversion, each member may have a different physicalread address. From the virtual read address, the actual byte number andMFI value can be identified easily for any follow up processing.

The addressing scheme is illustrated schematically in FIG. 2B. Referringbriefly to FIG. 2B, a group of three members 240-244 are shown. The J1byte for member 240 is at byte 130, the J1 byte for member 242 is atbyte 2103, and the J1 byte for member 2 is at byte 1027.

The current physical write addresses for each of members 240, 242, 244is at byte 2105, which is indicated by hash marks in FIG. 2B. While inthis example the physical write address for members 240, 242, 244 is thesame, the physical write address may differ for members in the samegroup, e.g., due to different SONET pointer movements from differentmembers. In this example:WAPhy(0)=2105WAPhy(1)=2105WAPhy(2)=2105

The virtual write addresses for the respective members may be expressedin the format (MFI, byte number), as follows:WAVir(0)=(i,1975)WAVir(1)=(i,2)WAVir(2)=(i,1078)

For member 240, the byte at the current write address is byte number1975 in frame i, so it is denoted (i, 1975). Similarly, for member 242,the byte at current write address is the second byte in frame i, so itis denoted as (i, 2). And for member 244, the byte at the current writeaddress is byte number 1078 in frame i, so it is denoted as (i, 1078).Among the members 240, 242, 244, member 242 has the minimum virtualwrite address.

In the example presented in FIG. 2B the threshold between the group readaddress and the minimum write address of the group is 2. Thus, thecurrent read address is at J1 byte of frame i, that is:RAVir=(i,0)

A common virtual read address may be assigned to the whole group.However, the values of physical read address are different for differentmembers. Applying the relationship RAPhy (i)=WAPhy (i) (WAVir(i)−RAVir), the physical read addresses of the respective member may bedetermined, as follows:RAPhy(0)=130RAPhy(1)=2103RAPhy(2)=1027

The described addressing method can be applied to memory wider than onebyte, and also to external memory (e.g., SDRAM). FIG. 5 is a schematicillustration of an eight-byte wide memory 500 with N words in accordancewith an embodiment. Received data frames may be written continuouslyinto the memory. Hence, any byte of the memory may store a J1 byte of areceived data frame. The address translation described above forone-byte wide memory can be applied to an eight-byte wide memory.Further, the memory may be treated as a circular memory. Thus, when awrite operation reaches the final byte of word N−1, the write operationcan continue with the first byte of word zero.

FIG. 6 is a schematic illustration of one embodiment of a memoryarchitecture in which M bytes of SDRAM store 64 STS-3c members (e.g.,for an OC-192 device). Hence, an M-byte block of SDRAM 610 is dividedinto four banks identified by bank 0 (612), bank 1 (614), bank 2 (616),and bank 3 (618). Each of the four banks of SDRAM 610 stores data fromsixteen members. In the embodiment depicted in FIG. 6, bank 0 (612)stores data from the set of members (0, 4, 8 . . . 60), bank 1 (614)stores data from the set of members (1, 5, 9 . . . 61), bank 2 (616)stores data from the set of members (2, 6, 10, . . . 62), and bank 4(618) stores data from the set of members (3, 7, 11, . . . 63).Separating members into different banks in the SDRAM 610 improves theaccess efficiency when reading from and writing to SDRAM 610.

In one embodiment each bank 612, 614, 616, 618 may be divided intochunks of 1024 bytes, illustrated in table 630. This allocates 64 bytesof memory to each of the sixteen members assigned to the bank, asillustrated in table 640. In turn, each 64 byte memory allocation isdivided into eight words of memory, each of which is 8 bits in width, asillustrated in table 650. The memory may be treated as a circularmemory. Thus, when a write operation reaches the final byte of wordM/512-1, the write operation can continue with the first byte of wordzero.

Reference in the specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least animplementation. The appearances of the phrase “in one embodiment” invarious places in the specification may or may not be all referring tothe same embodiment.

Thus, although embodiments have been described in language specific tostructural features and/or methodological acts, it is to be understoodthat claimed subject matter may not be limited to the specific featuresor acts described. Rather, the specific features and acts are disclosedas sample forms of implementing the claimed subject matter.

What is claimed is:
 1. A method, comprising: receiving a plurality ofdata frames at a destination node in a communications system; storingpayload data from each of the received plurality of data frames in amemory, the plurality of data frames representing at least one virtuallyconcatenated data stream; recording a physical write address at whicheach of the plurality of received data frames is stored in the memory;recording a virtual write address for each of the plurality of receiveddata frames to a memory location, the virtual write address including acorresponding multiframe indicator value for each of the received dataframes and a byte number for the last received byte from the receiveddata frames; and reading a group of associated data frames identified bycorresponding ones of the multiframe indicators and the byte numberindicators, the reading being based on determining a minimum writeaddress from a plurality of physical write addresses in the group ofassociated data frames by comparing virtual write addresses of allmembers in the group.
 2. The method of claim 1, wherein the plurality ofdata frames is received contemporaneously from a plurality of differentcommunication channels coupled to the destination node.
 3. A methodcomprising: determining a minimum write address for a group of data fromeach of the virtual write addresses of each of the plurality of memberswithin the group; determining a virtual read address for the group usingthe determined minimum write address and subtracting a threshold valueequal to a write-to-read delay in memory from the minimum write address;determining a physical read address for the group using the virtual readaddress for the group; and reading frame data based on a determinationof the physical read addresses for each of the plurality of members ofthe group.
 4. The method of claim 3, further comprising determining theminimum read address by operations comprising: comparing each of thevirtual write addresses of each of the plurality of members within thegroup; and selecting an address having the lowest value of each of thevirtual write addresses.
 5. The method of claim 3, further comprisingexpressing the minimum write address using a multiframe indicator and abyte number.
 6. The method of claim 3, further comprising determiningthe physical read address for each of the plurality of members withinthe group by subtracting the difference between a virtual write addressfor a selected one of the plurality of members and the virtual readaddress for the group from the physical write address of the selectedone of the plurality of members.
 7. The method of claim 3, furthercomprising determining multiframe indicator values and actual bytenumbers for each of the plurality of members within the group fromphysical read addresses for each of the plurality of members.
 8. Amethod, comprising: recording, for each of a plurality of data frames, avirtual write address including a multiframe indicator and a byte numberindicator; and reading a group of associated data frames identified bycorresponding ones of the multiframe indicators and the byte numberindicators, the reading being based on determining a minimum writeaddress from a plurality of physical write addresses in the group ofassociated data frames by comparing virtual write addresses of allmembers in the group.
 9. A computer-readable storage medium having notransitory signals, the computer-readable storage medium comprising aplurality of instructions that, when executed by one or more processorsconfigures the one or more processors to perform operations forcompensating for differential delays in communications, the operationscomprising: receiving a plurality of data frames at a destination nodein a communications system; storing payload data from each of thereceived plurality of data frames in a memory, the plurality of dataframes representing at least one virtually concatenated data stream;recording a physical write address at which each of the plurality ofreceived data frames is stored; reading contemporaneously from thememory a group of the received data frames, each of the received dataframes being identified by corresponding multiframe indicators and bytenumber indicators, the reading being based at least partially ondetermining a minimum write address from a plurality of physical writeaddresses in the group of the received data frames by comparing virtualwrite addresses of all members in the group; and recording a virtualwrite address for each of the plurality of received data frames to amemory location.
 10. The computer-readable storage medium of claim 9,further comprising one or more instructions that, when executed by theone or more processors, further configures the one or more processors torecord, for each of the plurality of data frames, the plurality ofphysical write addresses that each indicate a position in the memory.11. The computer-readable storage medium of claim 9, wherein readingcontemporaneously from the memory comprises: determining a virtual readaddress based on the minimum write address; and determining a physicalread address for each member in the group of associated data framesbased on the virtual read address.
 12. A method comprising: determininga minimum write address for a group of data from each of the virtualwrite addresses of each of the plurality of members within the group;determining a virtual read address for the group using the determinedminimum write address; determining a physical read address for the groupusing the virtual read address for the group and subtracting thedifference between a virtual write address for a selected one of theplurality of members and the virtual read address for the group from thephysical write address of the selected one of the plurality of members;and reading frame data based on a determination of the physical readaddresses for each of the plurality of members of the group.